Method and apparatus for automated image correction for digital image acquisition

ABSTRACT

A method and apparatus for automatically correcting colors in digital image space comprises taking an image file of the object using a digital device, such as a digital camera or scanner. One embodiment of the invention involves registering to obtain access to a profiling software. When registered, one obtains a target test sheet and accompanying manuals. The registration itself involves using a password or access code for using the software on the computer. The profiling software checks the image file generated from the printer or the digital camera and compares it to the photograph of the target test sheet under specific lighting conditions. The profiling software specifically checks the submitted object file for orientation, alignment, color intensity, and hue with respect to the target test sheet. When the image file is uploaded, a transformation image algorithm matches colors of the submitted image file with colors of the target test sheet and thereby creates a color corrected image profile for the submitted image file. Alternatively, the invention may be implemented entirely in an imaging device, such as a digital camera.

TECHNICAL FIELD

[0001] The invention relates to methods for assuring correct colors in adigital image file as a picture is captured by a camera, scanned, orprinted onto a medium at a specified printer location, or simplydisplayed in the digital space. More particularly, the invention relatesto methods for transforming the colors in a digital image to a colorcorrected digital image.

DESCRIPTION OF THE PRIOR ART

[0002] The traditional method for creating photographic images was basedupon the silver halide film process. A camera holds a film that containssilver halide particles which are sensitive to light. When thephotographer takes a picture, the image is recorded on the silver halideparticles, which are later chemically processed. To make prints,especially enlargements, the photographic image is then transferred viaa second photographic process to paper containing a layer of silverhalide particles, and the process is repeated. In the case of colorphotography, the film and the paper use silver halide for each coloralso but include tinted filters to assure that red light, for example,exposes one layer, blue light another layer, etc.

[0003] Different light sources have different balances of energy. As isquite well known in optics, visible light is made up of differentwavelengths of energy. Direct sunlight at midday is considered neutral;incandescent light is considered warm, because it has a higherproportion of red light; and fluorescent light is considered cool,because it has a higher proportion of green and blue light.

[0004] Although the eye sees these differences in light balance, thebrain generally compensates by adjusting the perceived color balance.Achieving a proper color balance in film and paper involves adjustingthe filters used on the separate layers of the silver halide crystals.Balancing a specific print is a painstaking, trial and error process.

[0005] Traditional still cameras do not affect color balance, except inthe trivial sense that lenses may have very slight chromatic effects.The film and paper have color characteristics that significantly affectthe color balance of the final print. Most color snapshots today areprinted using highly automated electronically controlled printingmachines that take these color profiles into account, and require littleif any user intervention to get pleasing color output.

[0006] However, these automated processes cannot completely compensatefor all variables, because a large portion of color variability isattributable to the light source used to illuminate the subject.Different sources of light include varying levels of the components ofthe spectrum of visible light, which causes them to appear to havedifferent overall color balances. Incandescent light tends to favoryellow; florescent light tends to favor bluish green. While the humanbrain tends to accommodate quickly to these differences and recalibratethe internal sense of what is a neutral white or gray, a photographcannot completely compensate one situation into another and the resultscan be very obvious even to non-professionals.

[0007] Color matching in professional photography is often of criticalimportance. Color matching in professional photography is generally timeconsuming because colors of various items, such as clothing and otherconsumer goods, must be precisely reproduced for catalogs,advertisements and brochures. The process could, therefore, getprohibitively expensive in most cases unless a photographer withexperience can expedite the processing.

[0008] In the last few years, digital photography has begun makingsignificant inroads as an alternative medium for creating andmanipulating images. In addition, the availability of inexpensivescanners has made it possible for many people to take photographscreated using traditional film processes and manipulate, transmit orprint them digitally.

[0009] Lately, digital color printing has also evolved significantly, inpart, because of color profiles created to map the color characteristicsof specific devices. It is, therefore, possible now to get consistentcolor output on a variety of devices. However, it is considerably moredifficult to assure proper color in the creation of a digital file, suchas a digital image file provided by a digital camera.

[0010] A common problem when one deals with digital color imagingdevices is getting the colors on the image as printed or displayed on amonitor to match the true object. For example, a color scanner as aninput device is associated with device dependent color space containingcolors such as red, green, and blue (RGB). The device dependent colorspace, where colors are described in digital RGB values, determines thecolor representation of that particular device. This color space isdifferent for each such imaging device and is thus called devicedependent color space.

[0011] Just as input devices are associated with device dependent colorspace, output devices, e.g. printers, also operate in device-dependentcolor space. Colors printed on any given printer typically do not matchcolors printed on a different printer because each printer is associatedwith a cyan, magenta, yellow, and black (CMYK) device dependent colorspace. Each image-rendering device, such as a printer or a monitor, hasa limited range of colors that it can reproduce, known as a gamut.

[0012] Under different lighting conditions, devices such as digitalcameras may also have difficulty accurately capturing the full range ofvalues of brightness or contrast. If the image does not accuratelydistinguish between darker values, then shadow detail is lost andpictures may appear muddy. If the image does not accurately distinguishbetween light areas, highlights appear without detail and the image maylook washed out. By using a test image with a range of values of thesame color, such as a grey scale, it is possible to adjust the range ofcontrast to achieve a more pleasing and natural color balance.

[0013] It would be advantageous to provide digital color correctionmethods to correct for the differences between the reference colors in acolor chart and those same colors as presented in an image that isrecorded by a photographic device or a film medium.

[0014] It would also be advantageous to provide automated colorcorrection methods to correct for the differences between the referencecolors in a color chart and those same colors as presented in an imagethat is recorded by a photographic device or a film medium, and toadjust for optimum contrast and brightness.

SUMMARY OF THE INVENTION

[0015] The invention provides color corrected digital profiling methodsthat can correct for differences between reference colors according to acolor chart and the reference colors as recorded and submitted as animage recorded by a photographic device or a film medium. In thepresently preferred embodiments, this process is automated so that thetime spent by a user of photo imaging devices is minimized and so thatspecial expertise is not required to achieve excellent color fidelityand contrast. In one presently preferred embodiment, this automatedprocess is encoded in software or hardware included within the camera orother image acquisition device.

[0016] One method in accordance with the invention comprises acquiring adigital image file of the test image or target containing color samplesof known values at known positions within the target using a digitalimage acquisition device such as a digital camera, scanner, or othersuitable device. The method further involves comparing the digital colorvalues of the test image with the known values to determine whether thetest image is properly aligned and oriented. The method further involvescomparing the known values of the colors in the test image with theactual values obtained by the device under the specific conditions inwhich it is being used. The method further involves generating acorrective color profile that adjusts the hue, contrast, and overallexposure of the image to match the reference values as closely aspossible. In one embodiment, this process takes place entirely withinsoftware and/or hardware contained within the image acquisition device.In another embodiment, the method involves use of software loaded on acomputer such as a PC. In a third embodiment, the method involvesregistering over a network such as the Internet to obtain access toprofiling software. The registration itself involves using a password oraccess code for using the software on the computer. In the presentlypreferred embodiment, the test image is submitted to a server upon whichthe color correction software is stored. The profiling software itselfchecks the image file generated from the scanner or the digital cameraand compares the values obtained with the known reference values of thecolors on the target test sheet under specific lighting conditions. Theprofiling software specifically checks the submitted object file fororientation, alignment, color intensity, and hue with respect to theimage of the target test sheet.

[0017] In each of the preferred embodiments, an image transformationalgorithm is created to match colors of the submitted image file withcolors of the target test sheet and thereby creates a color correctedimage profile for the submitted image file.

[0018] The image transformation algorithm matches the colors of thesubmitted image file with colors of the target test sheet using a binarycolor coding in which a unique binary number represents each color.Where the color correction process is contained within the imageacquisition device, the image transformation algorithm is used tocorrect subsequent images taken under the same lighting conditions.Where the color correction process is in software in a local computer oron a network, the user can apply the corrective profile to other imagesselected by the user. the generated color corrected image can bemanipulated, transmitted to a local or a remote printer, transmitted toa digital exhibit space, or simply compared to the submitted digitalimage. The generated color corrected image profile is typicallytransmitted in digital space via e-mail.

[0019] The color corrected image profile can be saved for future use oruploaded to a Web site wherein several image files can be colorcorrected in sequence or in random order.

BRIEF DESCRIPTION OF THE DRAWINGS

[0020]FIG. 1 is an illustration of the components in a color matchingtechnique according to the invention;

[0021]FIG. 2 is a flow diagram of the steps involved in profilingsoftware for the formation of color corrected digital image filesaccording to the invention;

[0022]FIG. 3A is a flow diagram of the steps involved in obtainingdigital color images using a color matching technique on a localprinting machine according to the invention;

[0023]FIG. 3B is a flow diagram of the steps involved in obtainingdigital color images using a color matching technique on a remoteprinting machine according to the invention;

[0024]FIG. 4A is a flow diagram of the steps involved in uploadingimages onto a digital exhibit space using a color matching techniqueaccording to the invention; and

[0025]FIG. 4B is a flow diagram of the steps involved in obtainingaccess to color images on a digital exhibit space using a color matchingtechnique according to the invention.

DETAILED DESCRIPTION

[0026] The invention provides a solution to the problem of colormatching of digital images using a color matching technique involving acolor chart for profiling, and profiling software that generates animage profile after correcting for the differences between the colorchart of known hue, saturation, and intensity with a recorded imagefrom, for example, a digital camera. The generated profile can be storedand viewed in an exhibit space or used for processing multiple images ofcolor corrected digital files.

[0027] To use the herein described color matching technique (FIG. 1),the user is provided with a target test sheet 2 and an imaging device,such as a digital camera 4 or a color scanner 6. To generate a colorprofile for a digital camera 4, the user takes a photograph of thetarget test sheet 2 under the specific lighting conditions for which theprofile is desired. In the case of a scanner 6, the user scans thetarget test sheet 2.

[0028] The following steps are same for both the scanner 6 and thedigital camera 4. The user then downloads the file containing an imageof the target to a client computer 8. The user then transmits the fileover a network such as the Internet 10 to a server 12. The server 12then opens the file 14.

[0029]FIG. 2 is a flow diagram showing the steps involved in profilingsoftware for forming color corrected digital image files. To use theprofiling software in one embodiment of the invention, one must beregistered and logged in to a Web site. Once the user is registered 22and logged in, the user can access software that allows the user toapply the transformation algorithms available at the Web site. Ideally,the user is able to accomplish these transformations through a one-clickprocess. After downloading the application, the user uploads a file 24,which the Web site logs with the user's unique identifier.

[0030] The Web site then runs a program that uses a pattern recognitionalgorithm or algorithms 26 to assure that the image of the targetcaptured by a photographer is aligned 28 correctly so that, for example,the cyan target in the transmitted image is corrected to equal theoriginal cyan target, rather than the blue or green, which would resultin a major distortion of the color balance. There are two generalapproaches that could be taken to accomplish this. One approach is that,prior to submission of the image containing the target, the usersoftware could prompt the user to identify via mouse clicks the locationof the corners or other identifying locations in the target that thesoftware could use as a reference to align the image of the target andaccurately compare the values of each color sample in the submittedimage with the correct color image sample.

[0031] A more automated system operates without requiring the user totake this step. In the case of a color correction system containedwithin the image acquisition device itself, the device, such as acamera, accomplishes this result without requiring the user toparticipate other than to inform the camera through an input, such asselection of a menu item or pressing of a button or other input device.The color correction software then samples the image, searching, forexample for the 100% cyan region in the test image by determiningwhether there is a region in the image that is roughly the appropriateshape and with color values relatively close to cyan. The softwaresamples a large number of regions to find a region that meets theselection criteria. Once the software identifies a region that appearsto satisfy the selection criteria. The software identifies such aregion, it searches the areas immediately adjacent to the identifiedregion for a color sample known to be adjacent to the that color samplein the test image. If the software does not locate the required colorsample candidate, it rejects the original sample and starts the processagain. If the software does locate the expected second color sample, itrepeats the process in searching for a third known sample, and so onthrough sufficient iterations to achieve a sufficiently high level ofconfidence that the target has been properly located and oriented. Suchimage recognition and pattern recognition software are known in the art.

[0032] The software then executes a transformation algorithm thatcompares the numerical value of specific color samples in the imagecaptured by the user to the known values in the original target, andgenerates a numerical difference value. For example, in the CMYK space,the proper value for a cyan area in the target image might be 255, 0, 0,0, where the first value represents the level of cyan on a scale of 0 to255, the second the level of magenta, the third of yellow, and the lastof black. If for example, the software determined that the value for thearea as captured by the acquisition device was 250, 10, 2, 5, theprogram would compare the submitted values to the known values andgenerate a transformation profile of +5, −10, −2, −5 to correct thecolor imbalance and thus match the color balance of the original image30 as closely as possible. The program then creates a profile, whichconsists of the settings or adjustments needed by the software toconvert an image with the color balance 32 in the submitted image to onehaving accurate color.

[0033] In the case of software or hardware included within the imageacquisition device, the device then uses the profile for subsequentimages. The image acquisition device uses the new profile until the userinputs his desire to stop using it. Alternatively, the device could useadditional software to sense that the lighting conditions have changed,discontinue use of the profile, and prompt the user to repeat theprofiling process. In the case of image correction on a local computeror over a network, a file containing the profile is created andoptionally saved.

[0034] In the case of software on a local computer, the user can thenapply the profile to one or more images. In the case of softwareoperating over a network, the server then transmits the profile, eithervia email or by allowing the user to otherwise download the profile. Inthe network embodiment of the invention, the server 12 (FIG. 1) sendsthe profile back to the client computer 8 over a network 10. The usercan then use the software 16 to load the profile onto the clientcomputer and apply the profile 14 to other image files, thus correctingcolor balance of those images. Ideally, this function is as simple asthe user using a mouse or other cursor controller to drag the profile 14icons onto an image file, or vice versa. The user then has the option toaccept or reject the corrected image and save the corrected file 14.This procedure can be repeated several times, for example at acustomer's request, until terminated, E.G. herein disclosed by thecustomer 38 (FIG. 2).

[0035] A photographer wishing to use the herein disclosed color matchingtechnique first obtains a target test sheet 2 (FIG. 1), which may be aprinted cardboard, plastic, paper, or other test sheet containing acolor target (a test pattern of color samples), as well as basicinstructions for use 50. He then uses his digital camera 4 to photographthe test sheet in the exact lighting conditions he wishes to control.Typically, this is accomplished by taking a test shot immediately before54 the picture he desires to calibrate. In the case of software embeddedin a digital camera, the user simply signals the camera to adjustexposure, and all subsequent photographs are automatically adjusted forthe current lighting conditions.

[0036] In the case of software available over the network, thephotographer then registers with the server or website 60. The server orweb site (FIG. 2, 22) gives the user a unique account. The photographerthen uploads the test photograph to the server or web site 62. The userreceives an executable file in the form of an email attachment or otherfile from the server or web site.

[0037] By clicking on the executable file, the user can then create aprofile on his computer that allows him to process images taken withthat camera and correct for the lighting conditions. For example, theuser could open the image using software such as Adobe Photoshop or aweb browser. Clicking on the desired icon automatically starts asoftware process that executes 64 the image transformation and allowsthe user to compare 66 the original and corrected images side-by-side onthe screen. The user can then choose 68 to accept 70 or reject 72 thecorrected image and save it in a new file 74. If the user has localaccess to a calibrated printer 76, he can then print 78 the images usingthe profile. If the user wishes to print remotely (FIG. 3B), he canupload the profiled print 102 to the server or web site in encryptedform and then specify a remote location to print 104 the file ascorrected by the algorithm herein. At the time the file is processed forprinting, this step is followed by the step of applying a profiling ofthe target printer 106 and terminating the session 110 as soon as theprinting is done 108. Printing may occur immediately after the file issent to the remote location, or may be delayed, for example, until theuser arrives at the remote printing location.

[0038] Alternately, the user can upload the corrected image to a digitalexhibit space linked to the server or web site (FIG. 4A and 4B). Theuser has the option to set permission and/or passwords to control theaccess to the digital images. FIG. 4A is a flow diagram that shows thesteps involved in uploading digital images onto a digital space usingthe herein disclosed color matching technique.

[0039] The user starts the uploading process 150 by obtaining a targettest sheet 152 and photographs the test sheet 156 in specific lightingconditions. Such lighting conditions could include, for example,incandescent lighting, fluorescent lighting, daylight, or anycombination thereof. The user then captures the image for whichcorrection is desired under the same lighting conditions 158.

[0040] In the case of a camera containing the color correction software,the user then instructs the camera to optimize the image. Alternatively,the camera could apply the transformation as a filter prior to capturingthe image, to maximize the physical capabilities of the device.

[0041] In the case of software accessible from a network, the user thenloads 162 the test photograph to the server or web site. The profilingsoftware color matches the values for the color target as captured inthe picture to the known values for those colors in the target testsheet from step 156. An executable file or, alternatively a profile tobe used by profiling software for subsequent processing is created bythe profile generating software and transmitted to the user via e-mail164 or other electronic means. The user can check one or more of hispictures for color matching by clicking or dragging the image onto theimage transformation icon 166 or, alternatively, by dragging the icon166 onto the image. The user can compare the original picture image withthe color corrected image on the screen 168 and check 170 to accept 174or reject 172 the corrected image file. If the user likes the colorcorrected image file, he may choose to create and access 176 a digitalexhibit space, wherein the mentioned digital exhibit space is linked tothe web site. The color corrected image file could also be uploaded 178to digital exhibit space for viewing instead of printing. The user canthen logoff to stop the viewing 180. A copy of the color corrected imagefile remains in the digital space created for the user (FIG. 4A) for useat a later time.

[0042] To log on to the digital exhibit space to obtain access todigital color images (FIG. 4B) at some later time, the user starts theprocess 200 by accepting an offer to log on to the digital image space202. It is determined then 208 if the user has a password, in which casehe provides one 210, or needs to obtain one 214. As soon as a passwordis provided, the user can view the images he created in digital imagespace 212 and log off when finished to stop 212 the process.

[0043] The presently preferred embodiment of the invention, therefore,provides a method of correcting colors in a digital image taken from animage generating device and a software that makes the color correctedimages available as either a downloadable image file or simply a filefor viewing in the digital space. It is expected to benefit a novicephotographer to a more experienced photographer in terms of time, costand quality of color print that is made.

[0044] Although the invention has been described in detail withreference to particular preferred embodiments, persons possessingordinary skill in the art to which this invention pertains willappreciate that various modifications and enhancements may be madewithout departing from the spirit and scope of the claims that follow.For example, a standard film cameras may incorporate software thatidentifies when a target is being imaged and use the camera's exposurecontrol sensors to capture ambient lighting information. Suchinformation is compared with profile information stored within thecamera and a correction value can be printed to the film for use by aprocessing laboratory. Further, while a software implementation of theinvention is described herein, it will be appreciated by those skilledin the art that the invention, or portions thereof, may comprise amechanism that is implemented in hardware level logic.

1. A method for correcting colors in digital image space, comprising:obtaining an image containing a specified target test sheet; creating animage file containing said image; providing a profiling mechanism wheresaid profiling mechanism optionally checking an image file for correctalignment with a target test sheet with information within said imagefile; said profiling mechanism matching colors of said image file withcolors of said target test sheet using a transformation image algorithm;and creating a color corrected image profile for said submitted imagefile.
 2. A method as in claim 1, wherein said method further comprisestransmitting said image profile.
 3. A method as in claim 1, wherein saidmethod further comprises using said image profile to correct color inanother image file.
 4. A method as in claim 1, wherein said methodfurther comprises using said image profile to correct contrast inanother image file.
 5. A method as in claim 1, wherein said registeringfor said profiling software further comprises obtaining a target testsheet and instructions for use.
 6. A method as in claim 1, wherein saiduploading an image file for submitting to said profiling mechanismfurther comprises photographing said target test sheet in specificlighting conditions.
 7. A method as in claim 1, wherein said uploadingan image file for submitting to said profiling mechanism furthercomprises photographing an object under suitable lighting conditions. 8.A method as in claim 1, wherein said profiling mechanism checking saidimage file for correct alignment further comprises checking for correctorientation with said target test sheet.
 9. A method as in claim 1,wherein said profiling mechanism checking said image file for correctalignment further comprises checking for correct intensity.
 10. A methodas in claim 1, wherein said profiling mechanism checking said image filefor correct alignment further comprises checking for correct hue.
 11. Amethod as in claim 1, wherein said transformation image algorithmmatches colors of said image file with colors of said target test sheetusing binary color coding such that each color is represented by aunique binary number.
 12. A method as in claim 1, wherein creating acolor corrected image profile for said submitted image file furthercomprises manipulating said color corrected image profile.
 13. A methodas in claim 1, wherein creating a color corrected image profile furthercomprises transmitting said color corrected image profile.
 14. A methodas in claim 1, wherein creating a color corrected image profile furthercomprises printing said color corrected image profile.
 15. A method asin claim 1, wherein creating a color corrected image profile furthercomprises linking said color corrected image profile to a digitalexhibit space.
 16. A method as in claim 1, wherein creating a colorcorrected image profile further comprises comparing said color correctedimage profile to said image file obtained from photographing an objectunder suitable lighting conditions.
 17. A method as in claim 1, whereincreating a color corrected image profile further comprises saving saidcolor corrected image profile for future viewing.
 18. A method as inclaim 7, wherein said uploading an image file for submitting to saidprofiling software further comprises uploading said image file to a website.
 19. A method as in claim 13, wherein creating a color correctedimage profile further comprises transmitting said color corrected imageprofile via e-mail.
 20. A method as in claim 14, wherein creating acolor corrected image profile further comprises printing said colorcorrected image profile on a local printer.
 21. A method as in claim 14,wherein creating a color corrected image profile further comprisesprinting said color corrected image profile at a specified remotelocation.
 22. A method as in claim 14, wherein creating a colorcorrected image profile further comprises printing said color correctedimage profile comprises applying a printer's device dependent colorspace for color correction.
 23. A method as in claim 15, whereincreating a color corrected image profile further comprises linking saiddigital exhibit space to said web site.
 24. A method as in claim 15,wherein creating a color corrected image profile further comprisesaccessing said digital exhibit space linked to said web site.
 25. Amethod as in claim 15, wherein creating a color corrected image profilefurther comprises accessing said digital exhibit space using accesscodes.
 26. A method as in claim 15, wherein creating a color correctedimage profile further comprises accessing said digital exhibit spaceusing a password.
 27. An apparatus for automatically correcting colorsin a digital image space, comprising: a processor; a profiling mechanismthat operates under control of said processor; a target test sheet; animage acquisition device for generating an image file for submission tosaid profiling mechanism; wherein said profiling mechanism optionallychecks said image file for correct alignment with said target testsheet; said profiling mechanism comprising a transformation imagealgorithm for matching colors of said image file with colors of saidtarget test sheet and for creating a color corrected image profile forsaid submitted image file.
 28. An apparatus as in claim 27, wherein saidtarget test sheet is digitized by acquiring said target test sheet withsaid image acquisition device under specific lighting conditions.
 29. Anapparatus as in claim 27, wherein said image acquisition devicecomprises a scanner.
 30. An apparatus as in claim 27, wherein said imageacquisitiondevice comprises a digital camera.
 31. An apparatus as inclaim 30, wherein said profiling mechanism is integrated into saidcamera.